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1 Abstract 



• Points of view or opinions stated in this 
document do not necessarily represent 
official OERI position or policy. 



This instalment development study had two goals. First, we had the pragmatic goal of 
developing a measurement tool that would be valid for research and useful for assessment and 
intervention in classrooms. Second, we sought to develop theory in the area of self-regulated 
learning. To accomplish these goals, we began with theory-driven ideas and existing 
instruments, and combined these with constructs that emerged from exploratory factor analysis. 
Our results indicate that metacognitive awareness and regulatory skills in a problem-solving 
context are comprised of five independent factors: Knowledge of Cognition, Objectivity, 
Problem Representation, Subtask Monitoring, and Evaluation. 

This research had two phases. In the first phase, we reviewed current techniques for 
measuring variables related to metacognition and self-regulation and decided on a self-report 
measure — mainly for pragmatic reasons. Next, data was collected using two existing 
inventories related to metacognition and problem solving. After eliminating items based on 
reliability analyses, remaining items were factor-analyzed resulting in five factors. In the second 
phase, a new inventory was developed focusing on these five factors. Again, reliability analyses 
were conducted, and the remaining items were factor-analyzed. 

Our results have important implications for education. They indicate that there are five 
particular metacognitive and self-regulatory constructs relevant to problem solving. Being able 
to identify and delineate these constructs further should allow our educational research and 
teacher professional development teams to begin providing teachers with a set of tools and 
training resources to help them promote student self-regulation in their classrooms. These 
resources would be important to teachers who are concerned not only about what students learn 
but also about how they learn it. 

Further, our analyses indicate that the constructs measured by the IMSR are independent , 
and therefore a student may show preferences or “styles” of metacognitive strengths and 
weaknesses that depend upon his or her unique combination of constmcts. If these “styles” can 
be further understood and delineated, it might be possible to train students to habitually use 
particular regulatory behaviors. 



1 1 Contact: Bruce Howard or Steven McGee. 316 Washington Avenue, Wheeling. WV 26003. (304) 243-2388, howarcl@cet.edu, 
mcgee@cet.edu 
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Background 

Self-Regulated Learning, Metacognition, and Problem Solving 

Various definitions and models of self-regulated learning abound in the research 
literature (Bruning, Schraw and Ronning, 1995). Zimmerman (1989) captured the sweeping 
purview of self-regulated learning when he said, “Students can be described as self-regulated to 
the degree that they are metacognitively, motivationally, and behaviorally active participants in 
their own learning process” (p.4). 

For this study, we concentrated on measuring what it means to be a “metacognitively 
active participant” in the process of problem solving. Metacognition has been referred to as 
knowledge and regulation of one’s own cognitive system (Brown, 1978; Palincsar & Brown, 
1987). Metacognition enables students to coordinate the use of current knowledge and a 
repertoire of reflective strategies to accomplish a single goal. Metacognitive awareness, 
therefore, serves a regulatory function and is essential to effective learning because it allows 
students to regulate numerous cognitive skills. Investigation of metacognition in academic 
settings has traditionally focused on two major components: knowledge of cognition, or how 
much learners understand about their own memory organization and the way they learn, and 
regulation of cognition, or how well learners can regulate their own memory and learning 
(Brown 1980; 1987). 

Schraw and Dennison developed a 52-item Likert scale self-report inventory for adults 
(the MAI, or Metacognitive Awareness Inventory), which measured both knowledge of 
cognition and regulation of cognition (Schraw & Dennison, 1994). Although Schraw and 
Dennison set out to confirm the existence of eight factors (three related to knowledge of 
cognition and five related to regulation of cognition), the final factor structure was best 
represented by two factors (accounting for 65% of the sample variance). Post-hoc content 
analyses confirmed that these factors were indeed knowledge of cognition and regulation of 
cognition. 

These two factors were found again in a second instrument development study. 

Dennison, Howard, Krawchuk, and Hill used the results from the MAI study to develop two 
inventories for use with younger learners, the Jr MAI, versions A & B, (Dennison, Howard, 
Krawchuk & Hill, 1996). Their results indicated that both versions A & B revealed two distinct 
factors, accounting for 64% and 56% of the sample variance respectively. 

Method 

We began with the pragmatic goal of developing an instrument that would further our 
research and could also be used extensively in classrooms across the country to help teachers 
identify students’ self-regulatory strengths and weaknesses. It was not our intention to replicate 
the work of our predecessors in this area. Instead, we wanted to develop an easy-to-use self- 
report inventory for use with 1 2- 18-year-olds that focused more specifically on metacognitive 
awareness and regulatory skills for solving mathematical and scientific problems. 

Phase One 

In Phase One the objectives were: (1) to confirm the existence of factors related to 
knowledge of cognition and regulation of cognition in the context of problem solving, and (2) to 
extend our understanding of regulatory skills related to planning, monitoring, and evaluating. To 
this end, we began with two existing public domain inventories, the Jr. MAP version B (discussed 
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previously) to examine constructs related to knowledge of cognition and regulation of cognition, 
and the How I Solve Problems inventory (HISP) to examine constructs related to regulatory 
skills of planning, monitoring, and evaluating. The HISP (Fortunato, Hecht, Tittle & 

Alvarez, 1991) is discussed in detail below. 

Procedure and Participants 

We had three groups of participants: 134 students enrolled in two summer computer 
camps near Pittsburgh, 85 students from California, and 120 students from Michigan. Together 
these totaled 339 students aged 10-19 (53.3% male, 45.7% female). Ethnic breakdown included 
70.3% Caucasian, 8.2% Asian American, 7.7% Hispanic, and 2.1% African American. 

Materials 

Jr. MAI form 2 (Version B) 

The Junior Metacognitive Awareness Inventory was developed by Dennison, Murphy, 
Howard, & Hill (1996). In version B, for grades 7-9, students are instructed to read 1 8 items, 
and for each item circle the answer that best described the way they are when doing schoolwork 
or homework (l=never, 2=seldom/rarely, 3=sometimes, 4=often/frequently, 5=always). 

How I Solve Problems 

The HISP was developed by Fortunato, Hecht, Tittle, & Alvarez (1991) and originally 
used 2 1 items, with a 3-point scale (yes, maybe, no). In addition, the inventory was subdivided 
into four sections: three sections about working the problem before, during, and after, and one 
section about particular strategies a student might use. For our use the HISP was adapted to be 
similar in style and form to the Jr. MAI— with a five-point scale instead (l=never, 
2=seldom/rarely, 3=sometimes, 4=often/frequentIy, 5=always) and some minor wording 
changes. Students were instructed to read 21 items and for each item circle the answer that best 
described the way they are when trying to solve a problem they might see in a math or science 
class. 



Results 

Item Analysis 

Both instruments were treated as a single inventory for our purposes. Item means ranged 
from 2.3 to 4.5, and standard deviations ranged from .71 to 1.1. The standardized coefficient 
alpha was .8569. Upon eliminating items II, 17, 18, and 20 from the HISP, the reliability rose to 
alpha = .891 1 . 

Factor Analysis 

We conducted an exploratory factor analysis with the expectation that we would see four 
factors: a knowledge of cognition factor from particular Jr.MAI items, and three regulation of 
cognition factors (planning, monitoring and evaluation) from both Jr.MAI items and the 
remaining HISP items (36 total items). For the purposes of extracting the most independent 
constructs, we utilized a principle components extraction method using varimax factor rotation. 

Initial results revealed an eight-factor solution with eigenvalues 1 .043 and above, which 
accounted for 55.2% of the sample variance. Visual inspection showed that factors 6, 7, and 8 
were comprised of only a total of four items that loaded .35 and above. A content analysis of 
these four items showed that they measured particular learning strategies (e.g., “I draw pictures 
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or diagrams to help me understand while learning”; “I write down important information”). 

These items were dropped, and the factor analyses were conducted again. The resulting solution 
revealed five factors accounting for 42.7% of the sample variance. 

Our next procedure was to establish face validity for the five factors and the items that 
comprised them. Face validity was an important goal since we wanted an inventory that could 
be widely used by classroom teachers. We conducted a content analysis of the remaining 32 
items. We wrote definitions for the five factors based upon the three or four items that loaded 
the most heavily on a factor (.5 and above), and used a team of five raters to determine which 
items they thought “hung with” the factor definitions. Each rater categorized all the items into 
the five categories, and we worked towards consensus on the items we thought best represented 
the five factors. We thus eliminated another nine items, resulting in a final set of 23 items to use 
for Phase Two. The final 23 items were again factor analyzed, and again five factors were found 
accounting for 56.3% of the sample variance. 

Phase Two 

In Phase Two, our goal was to create a new inventory specific to metacognitive 
awareness and regulatory skills in the context of problem solving. To this end, we examined the 
23 remaining items from the original two inventories and revised or rewrote them to increase 
reliability, and wrote additional items to clearly demonstrate the existence of the five factors that 
had emerged in Phase One. 

Procedures/ Participants 

We recruited 829 students from schools across the country in grades 6-12 to complete the 
revised inventory (5 1 % male, 49% female). The ethnic breakdown included 85.2% Caucasian, 
7.5% Hispanic, 5% African American, 1 .9% Asian American, and 0.8% Pacific Islander. 

We titled the new instalment the Inventory of Metacognitive Self- Regulation (IMSR). 
The IMSR included 37 items with a five-point Likert scale. For each of the 37 items, students 
were instructed to circle the answer that best described the way they are when doing schoolwork 
or homework (l=never, 2=seldom/rarely, 3=sometimes, 4=often/frequently, 5=always). The 
items are shown in Table I . 



Results 

The overall inventory demonstrated a reliability of alpha=.935. We conducted an 
exploratory principle components factor analysis using a varimax rotation. The resulting 
solution revealed five factors with eigenvalues over 1.12, which accounted for 5 1 .6% of the 
variance. Reliability for each factor ranged from alpha=.720 to alpha= .867. Table I shows the 
factors, their descriptions, and the factor weights above .40. In addition, Table 1 shows three 
items (asterisked) that weighed only moderately across several factors, or weighed heavily on 
factors different than those hypothesized. For future research we would recommend removing or 
revising these three items. 



Discussion 

We found Knowledge of Cognition to be an important factor in this instrument as it was 
in the Jr. MAI and the adult MAI (Dennison et. al., 1996; Schraw & Dennison, 1994). The fact 
that Knowledge of Cognition stands out so clearly and independently indicates that more work 
should be done in understanding this construct. Prior research on metacognition and self- 



5 



AERA 2000 



5 



regulation has also discussed the importance of constructs similar to Monitoring and Evaluation, 
but very little research has connected the factors of Objectivity and Problem Representation to 
self-regulation. 

Our results have important implications for education. They indicate that there are five 
particular metacognitive and self-regulatory constructs relevant to problem solving. Being able 
to identify and delineate these constructs further should allow our educational research and 
teacher professional development teams to begin providing teachers with a set of tools and 
training resources to help them promote student self-regulation in their classrooms. These 
resources would be important to teachers who are concerned not only about what students learn 
but also about how they learn it. Our analyses indicate that the constructs measured by the IMSR 
are independent , and therefore a student may show preferences or “styles” of metacognitive 
strengths and weaknesses depending upon his or her unique combination of constructs. If these 
“styles” can be further understood and delineated, it might be possible to train students to 
habitually use particular regulatory behaviors. 
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Table 1 

The Five Factors. Their Descriptions and Facto r Weights Above .40 





factor 1 


factor 2 


Factor 3 


Factor 4 


Factor 5 


Knowledge of Cognition: How much learners understand about the extent and 
utilization of their unique cognitive abilities and the ways they learn best 












l can make myself memorize something. 










0.6952 


I use different ways to memorize things. 










0.6493 


When it comes to learning, I can make myself learn when 1 need to. 










0.6350 


When it comes to learning, 1 know how I learn best. 










0.6282 


I use learning strategies without thinking. 










0.5000 


When It comes to learning, 1 know my strengths and weaknesses. 










0.4060 


Objectivity: Standing outside oneself and thinking about one s learning as It 
proceeds 












When 1 am done with my schoolwork, 1 ask myself If I learned what I wanted to 
learn 






0.7006 






1 think about how well I am learning when I work a difficult problem. 






0.6793 






1 ask myself if there are certain goals I want to accomplish. 






0.6384 






1 ask myself how well I am doing while 1 am learning something new. 






0.6056 






1 think of several ways to solve a problem and then choose the best one. 






0.4938 






I try more than one way to learn something. 






0.4842 






M ask myself what is the easiest way to do things. 


.0269 


0.4926 


0.2344 


.1450 


.2093 


Problem Representation: Understanding the problem fully before proceeding 












I try to understand what the problem is asking me. 








0.6406 




1 read the problem more than once. 








0.6322 




1 l ead the problem over and over until I understand it. 








0.5842 




I think to myself, do 1 understand what the problem is asking me? 








0.5582 




I try to understand the problem so 1 know what to do. 








0.4615 




*1 make an extra effort to pay attention to what's important. 


0.3496 


0.3252 


0.1658 


0.2970 


0.0668 


Subtask Monitoring Breaking the problem down into subtasks and monitoring the 
choice of learning strategies and completion of each subtask 












I use different learning strategies depending on the problem. 




0.G595 








1 use different ways of learning depending on the problem. 




0.574G 








I try to break down the problem to just the necessary information. 




0.5530 








I identify all the Important parts of the problem. 




0.5335 








1 (ry to eliminate information in the problem that 1 don't need. 




0.5025 








1 pick out the steps 1 need to do this problem. 




0.4G50 








1 think about what information I need to solve this problem. 




0.44GG 








’1 think about all the steps as I work the problem. 


0.3871 


0.3868 


0.3110 


0.3566 


0.1446 


Evaluation: Double-checking throughout the entire problem-solving process to 
evaluate if it is being done correctly 












1 go back and check my work. 


0,7786 










1 double-check to make sure I did it right. 


0.7486 










1 check to see if my calculations are correct. 


0.6942 










1 look back to see if l did the correct procedures. 


0.G592 










1 check my work all the way through the problem. 


0.G34G 










1 look back at the problem to see if my answer makes sense. 


0.5888 










1 stop and rethink a step I have already done. 


0.577G 










1 make sure I complete each step. 


0.4591 











*asterisked items weighed only moderately across several factors, or weighed heavily on 
factors different than those hypothesized. 
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How do You Solve Problems? 



Please read the following sentences and circle the answer that best describes the way you are when 
you are trying to solve a problem. Think about a problem that you might see in a science or math 
class. 

• Think about when you have to solve a hard problem. What do you do before you start? 

• What do you do while you work the problem? 

• What do you do after vou finish working the problem? 



There are no right answers— please describe yourself as you are, not how you want to be or think you 
ought to be. Your teacher will not grade this. 

Never Seldom/ Rarely Sometimes Often/ Frequently Always 

A B C D E 



1 . I try to understand what the problem is asking me. 

2. I think of several ways to solve a problem and then choose the best one. 

3. I look back at the problem to see if my answer makes sense. 

4. I use different ways to memorize things. 

5. I think to myself, do I understand what the problem is asking me? 

6. I read the problem more than once. 

7. I think about what information I need to solve this problem. 

8. I use different learning strategies depending on the problem. 

9. I look back to see if I did the correct procedures. 

10. I think about how well I am learning when I work a difficult problem. 

11.1 use different ways of learning depending on the problem. 

12. I go back and check my work. 

13. I read the problem over and over until I understand it.. 

14. For this question, please circle letter B. 

15. I check to see if my calculations are correct. 

16. When it comes to learning, I can make myself learn when I need to. 



A B C D 
A B C D 
A B C D 
A B C D 
A B C D 
A B C D 
A B C D 
A B C D 
A B C D 
A B C D 
A B C D 
A B C D 
A B C D 
A B C D 
A B C D 
A B C D 



E 

E 

E 

E 

E 

E 

E 

E 

E 

E 

E 

E 

E 

E 

E 

E 




f 



Never Seldom/ Rarely Sometimes Often/ Frequently Always 

ABC D E 



17. I ask myself how well I am doing while I am learning something new. 

18. I check my work all the way through the problem. 

19. I identify all the important parts of the problem. 

20. I try to understand the problem so I know what to do. 

21. I think about all the steps as I work the problem. 

22. I can make myself memorize something. 

23. When it comes to learning, I know my strengths and weaknesses. 

24. I pick out the steps I need to do this problem. 

25. When I am done with my schoolwork, I ask myself if I learned what I wanted to 
learn. 

26. I double-check to make sure I did it right. 

27. For this question, please circle letter A. 

28. I try to break down the problem to just the necessary information. 

29. I use learning strategies without thinking. 

30. When it comes to learning, I know how I learn best. 

31 . I ask myself if there are certain goals I want to accomplish. 

32. I try more than one way to learn something. 



A B C D 
A B C D 
A B C D 
A B C D 
A B C D 
A B C D 
A B C D 
A B C D 
A B C D 

A B C D 
A B C D 
A B C D 
A B C D 
A B C D 

A B C D 
A B C D 



The Inventory of Metacognitive Self-Regulation ©. Copyright 1998-2000 by Wheeling Jesuit University/ Center for 
Educational Technologies. All rights reserved. 

First presented in Howard, B. C., McGee, S., Sliia, R., & Hong, N. S. (2000, April). Metacognitive Self-Regulation ancl 
Problem-Solving: Expanding the Theory Base Through Factor Analysis. Paper presented at the Annual Meeting of the 
American Educational Research Association, New Orleans, LA. 
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